Summary

Closes #245181

Fleet currently supports user customization at the type level (e.g. logs@custom) and data-stream level (e.g. logs-system.application@custom), but not at the namespace level. Users managing multiple integrations under a shared namespace must configure each data stream individually.

This PR implements namespace level customization by adding opt-in per-namespace index templates.

This is an API-only solution for now. #264065 will implement UI changes.

Solution rationale

Fleet currently creates one base index template per data stream pattern defined by the integration, e.g. logs-system.application-*. This index templates applies to all matching data streams regardless of namespace, e.g. logs-system.application-production and logs-system.application-staging). This means that namespace-specific component templates (e.g. production@custom and staging@custom) cannot belong to that index template, otherwise they would compete with each other and affect all data streams, not just the relevant namespace.

In this change, Fleet creates a dedicated index template per data stream with a more specific index pattern and higher priority than the base template. This namespace template is a clone of the base template with {namespace}@custom added to composed_of. The base template is never modified.

Example:

Component templates in logs-system.application:

"composed_of": [
  "logs@mappings",
  "logs@settings",
  "logs-system.application@package",
  "logs@custom",
  "system@custom",
  "logs-system.application@custom",
  "ecs@mappings",
  ".fleet_globals-1",
  ".fleet_agent_id_verification-1"
],

Component templates in logs-system.application@namespace.production:

"composed_of": [
  "logs@mappings",
  "logs@settings",
  "logs-system.application@package",
  "logs@custom",
  "system@custom",
  "production@custom", // namespace level, between package and data stream
  "logs-system.application@custom",
  "ecs@mappings",
  ".fleet_globals-1",
  ".fleet_agent_id_verification-1"
],

Opt-in mechanism

Creating namespace templates for every namespace by default could potentially cause many templates to be created unnecessarily. To mitigate this, namespace level customization is opt-in and managed in the package installation:

{
  "item": {
    "name": "system",
    ...
    "installationInfo": {
      ...
      "namespace_customization_enabled_for": ["production"]
      }
    }
  }
}

Namespaces can be opted in by updating the package installation:

PUT kbn:/api/fleet/epm/packages/system
{
  "namespace_customization_enabled_for": ["production"]
}

Or using the new bulk API:

POST kbn:/api/fleet/epm/packages/_bulk_namespace_customization
{
  "packages": ["system", "nginx", "apache"],
  "enable":  ["production"],
  "disable": ["staging"]
}

Space awareness

Installation saved objects are shared across spaces, meaning the list of opted in namespaces for an installed package is cluster wide: any Kibana space that can see an integration sees the same opt-in list. Index templates and component templates are also cluster wide.

The present implementation attempts to mitigate the write-side of this using the allowed_namespace_prefixes per-space restriction mechanism in Fleet settings (which is already used to gate which namespaces a user can choose for agent/package policies in a given space).

When the user attempts to modify the list of opted in namespaces, it will be validated against allowed namespace prefixes if any (if none are set, there's no validation). For example, if prod namespace is allowed in space A, then from that space prod_eu and prod_us would be allowed, but not qa.

Performance and scaling

Namespace template creation and deletion is handled by an asynchronous task. This addresses risks of latency/timeouts and provides a built-in retry mechanism.

Testing

1. Per-package opt-in creates namespace templates (existing policies)

   1. Install the System integration with a package policy using namespace production.

   2. Verify no namespace template exists yet:

   GET _index_template/logs-system.application@namespace.production
   

   should return 404.

   3. Opt in production:

   PUT kbn:/api/fleet/epm/packages/system
     {
       "namespace_customization_enabled_for": ["production"]
     }
   

   4. Wait a few seconds for the async task, then verify the namespace index template was created:

   GET _index_template/logs-system.application@namespace.production
   

   In particular, check:

   • index_patterns is ["logs-system.application-production*"]
   • priority is 250
   • composed_of includes production@custom

2. Namespace template shows in Assets tab

   In Kibana UI → Integrations → System → Assets tab, confirm logs-system.application@namespace.production (and any other per-dataset variants) appear alongside other Fleet-managed assets.

3. Namespace component templates are applied to their specific data streams only

   1. Create a production@customcomponent template, e.g.:

   PUT _component_template/production@custom
   {
     "template": {
       "settings": {
         "index.number_of_replicas": 2
       }
     }
   }
   

   2. Trigger a rollover if you already have data, or ingest a document to create backing indices, e.g.

   POST logs-system.application-production/_doc
   {
     "@timestamp": "2026-04-15T00:00:00Z",
     "message": "test document"
   }
   

   This should create the data stream and its first backing index automatically, using the logs-system.application@namespace.production index template (priority 250) rather than the base logs-system.application template (priority 200).
   3. Verify the production namespace data stream has number_of_replicas: 2:

   GET logs-system.application-production/_settings/index.number_of_replicas
   

   4. Verify a data stream in a different namespace does not have number_of_replicas: 2, confirming namespace isolation.

4. Opt-out deletes index templates but keeps component templates

   1. Opt out production:

   PUT kbn:/api/fleet/epm/packages/system
     {
       "namespace_customization_enabled_for": []
     }
   

   2. Wait a few seconds for the async task, then verify the namespace index templates were deleted but the production@customcomponent template still exists.

5. Uninstall cleans up namespace templates

   1. Opt production back in, namespace index template should exist.
   2. Delete all System policies, uninstall integration and verify the namespace index templates were deleted.

6. Bulk endpoint

   1. Install System and Nginx and enable production and staging namespaces for both:

   POST kbn:/api/fleet/epm/packages/_bulk_namespace_customization
   {
     "packages": ["system", "nginx"],
     "enable": ["production", "staging"]
   }
   

   2. The response should be:

   {
     "items": [
       {
         "name": "system",
         "success": true,
         "namespace_customization_enabled_for": [
           "production"
         ]
       },
       {
         "name": "nginx",
         "success": true,
         "namespace_customization_enabled_for": [
           "production"
         ]
       }
     ]
   }
   

   3. Test combinations of enable and disable.
   4. Verify that passing the name of a non installed package results in error:

   {
     "items": [
       {
         "name": "system",
         "success": true,
         "namespace_customization_enabled_for": [
           "production"
         ]
       },
       {
         "name": "apache",
         "success": false,
         "error": "Package apache is not installed"
       }
     ]
   }
   

   5. Verify that it is not allowed to pass the same namespace in enable and disable:

   {
     "statusCode": 400,
     "error": "Bad Request",
     "message": "Namespaces must not appear in both enable and disable: production"
   }
   

7. Space awareness

   1. Create a custom space and go to it.
   2. Configure allowed namespace prefixes in that space:

   PUT kbn:/api/fleet/space_settings
   { 
     "allowed_namespace_prefixes": ["prod"] 
   }
   

   3. Verify that you now can't opt in a namespace that doesn't match the allowed prefix:

   PUT kbn:/s/team-a/api/fleet/epm/packages/system
   { 
     "namespace_customization_enabled_for": ["staging"] 
   }
   

   Should result in:

   {
     "statusCode": 400,
     "error": "Bad Request",
     "message": "Cannot change namespace customization for: staging. Allowed prefixes in this space: prod"
   }
   

   4. Repeat with a matching namespace (e.g. prod_eu), it should work.
   5. Now change the allowed namespace prefixes to staging and try to opt out prod_eu (pass an empty array): it should reject with Cannot change namespace customization for: prod_eu. Allowed prefixes in this space: staging.
   6. Do similar checks with the bulk endpoint: note that if you pass multiple namespaces to opt in/out and at least one of them fails for a given package, the update will fail for this package.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Risk of eventual consistency gap if package installation SO is saved (synchronous) but the async task fails to create/delete index templates.

Release note

Add opt-in namespace level customization to integrations.